Hand Poste Estimation with Constrained Multi-hypotheses Gradient-Descent

In this report, we detail a novel approach to recover 3D hand pose from 2D images. To this end, we introduce a compact 3D hand model in a low dimension space where anatomy, kinematics and dynamics are implicitly inherited. The parameters of this model are recovered through a Bayesian inference approach. To this end, we propose an objective function which aims at separating the hand-skin characteristics within the 2D hand silhouette from the cluttered background. To address computational issues a polygonal approximation of the silhouette is considered and the differentiations from the 3D model to the 2D silhouette projection are carried out. Optimization of the cost function is done through a smart particle filtering approach which combines classical particle filters and local search. We further develop this concept towards reducing the number of hypotheses to be tested while retaining its performance through the use of a constrained variable metric gradient descent step. Very promising experimental results demonstrate the potentials of our approach.